A Memory-Based Approach to Anti-Spam Filtering

This paper presents an extensive empirical evaluation of memory-based learning in the context of anti-spam filtering, a novel cost-sensitive application of text categorization. Unsolicited commercial e-mail, also known as “spam”, floods the mailboxes of users, causing frustration, wasting bandwidth and money, and exposing minors to unsuitable content. Using a recently introduced publicly available corpus, a thorough investigation of the effectiveness of a memory-based anti-spam filter is performed, including different attribute and distance weighting schemes, and studies on the effect of the neighborhood size, the size of the attribute set, and the size of the training corpus. Three different cost scenarios are identified, and suitable cost-sensitive evaluation functions are employed. We conclude that memory-based anti-spam filtering is practically feasible, especially when combined with additional safety nets. Compared to a previously tested Naïve Bays filter, the memory-based filter performs on average better, particularly when the misclassification cost for non-spam messages is high.